Originally, Religious Knives started in Brooklyn, New York, United States in 2005 as a two-piece side project of husband and wife Mike Bernstein and Maya Miller of the improvisational band Double Leopards. The duo decided to branch out to pursue music in a new incarnation, and after adding Nate Nelson from Mouthus on drums they started to realize the new band's potential. Bernstein and Miller pulled out of Double Leopards and roped long time friend Todd Cavallo in a bass player, after a few auditions.